Homeopathy can offer empirical insights on treatment effects in a null field.

OBJECTIVES: A "null field" is a scientific field where there is nothing to discover and where observed associations are thus expected to simply reflect the magnitude of bias. We aimed to characterize a null field using a known example, homeopathy (a pseudoscientific medical approach based on using highly diluted substances), as a prototype. STUDY DESIGN AND SETTING: We identified 50 randomized placebo-controlled trials of homeopathy interventions from highly cited meta-analyses. The primary outcome variable was the observed effect size in the studies. Variables related to study quality or impact were also extracted. RESULTS: The mean effect size for homeopathy was 0.36 standard deviations (Hedges' g; 95% confidence interval: 0.21, 0.51) better than placebo, which corresponds to an odds ratio of 1.94 (95% CI: 1.69, 2.23) in favor of homeopathy. 80% of studies had positive effect sizes (favoring homeopathy). Effect size was significantly correlated with citation counts from journals in the directory of open-access journals and CiteWatch. We identified common statistical errors in 25 studies. CONCLUSION: A null field like homeopathy can exhibit large effect sizes, high rates of favorable results, and high citation impact in the published scientific literature. Null fields may represent a useful negative control for the scientific process.

Redundant meta-analyses are common in genetic epidemiology.

OBJECTIVES: The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology. STUDY DESIGN AND SETTING: Using a sample of 38 index meta-analyses of genetic associations published in 2010, we retrieved additional meta-analyses that evaluated identical associations (same genetic variant and phenotype) using the Human Genome Epidemiology (HuGE) Navigator and PubMed databases. We analyzed the frequency of potential duplication and examined whether subsequent meta-analyses cited previous meta-analyses on the exact same association. RESULTS: Based on 38 index meta-analyses, we retrieved a total of 99 duplicate meta-analyses. Only 12 (32%) of the index meta-analyses were unambiguously unique. We found a mean of 2.6 duplicates and a median of 2 duplicates per meta-analysis. In case studies, only 29-54% of previously published meta-analyses were cited by subsequent ones. CONCLUSION: These results suggest that duplication is common in meta-analyses of genetic associations.